Goto

Collaborating Authors

 long-tail problem


Deep-Learning-Assisted Highly-Accurate COVID-19 Diagnosis on Lung Computed Tomography Images

Wang, Yinuo, Bae, Juhyun, Chow, Ka Ho, Chen, Shenyang, Gupta, Shreyash

arXiv.org Artificial Intelligence

-- COVID-19 is a severe and acute viral disease that can cause symptoms consistent with pneumonia in which inflammation is caused in the alveolous regions of the lungs leading to a build-up of fluid and breathing difficulties. Thus, the diagnosis of COVID using CT scans has been effective in assisting with RT -PCR diagnosis and severity classifications. In this paper, we proposed a new data quality control pipeline to refine the quality of CT images based on GAN and sliding windows. Also, we use class-sensitive cost functions including Label Distribution A ware Loss(LDAM Loss) and Class-balanced(CB) Loss to solve the long-tail problem existing in datasets. Our model reaches more than 0.983 MCC in the benchmark test dataset. I. INTRODUCTION Severe acute respiratory syndrome coronavirus 2 (SARS-CoV -2) infection still plays a major role in world policy changes and continues to effect billions every day. The severity of the infection rises with each day and early diagnosis can be crucial for disease control. Pandemic on such a massive scale which has impacted about 83 million people in US itself, has put primary methods of diagnosis via RT -PCR under stress. A reliable COVID-19 classification method is needed to relieve the pressure of manual clinical diagnostics.


LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation

Liu, Qidong, Wu, Xian, Wang, Wanyu, Wang, Yejing, Zhu, Yuanshao, Zhao, Xiangyu, Tian, Feng, Zheng, Yefeng

arXiv.org Artificial Intelligence

Sequential Recommender Systems (SRS), which model a user's interaction history to predict the next item of interest, are widely used in various applications. However, existing SRS often struggle with low-popularity items, a challenge known as the long-tail problem. This issue leads to reduced serendipity for users and diminished profits for sellers, ultimately harming the overall system. Large Language Model (LLM) has the ability to capture semantic relationships between items, independent of their popularity, making it a promising solution to this problem. In this paper, we introduce LLMEmb, a novel method leveraging LLM to generate item embeddings that enhance SRS performance. To bridge the gap between general-purpose LLM and the recommendation domain, we propose a Supervised Contrastive Fine-Tuning (SCFT) approach. This approach includes attribute-level data augmentation and a tailored contrastive loss to make LLM more recommendation-friendly. Additionally, we emphasize the importance of integrating collaborative signals into LLM-generated embeddings, for which we propose Recommendation Adaptation Training (RAT). This further refines the embeddings for optimal use in SRS. The LLMEmb-derived embeddings can be seamlessly integrated with any SRS models, underscoring the practical value. Comprehensive experiments conducted on three real-world datasets demonstrate that LLMEmb significantly outperforms existing methods across multiple SRS models. The code for our method is released online https://github.com/Applied-Machine-Learning-Lab/LLMEmb.


Reviews: Learning to Model the Tail

Neural Information Processing Systems

Summary ------- The paper proposes an approach for transfer learning for multi-class classification problems that aids the learning of categories with few training examples (the categories in the tail of the distribution of numbers of examples per category). It is based on ideas of meta-learning: it builds a (meta-)model of the dynamics that accompany the change in model parameters as more training data is made available to a classifier. Specifically, the proposed approach takes inspiration from existing work on meta-learning [20] but extends it by applying it to CNNs, utilizing deep residual networks as the meta-model, and applying the framework to a general'long-tail problem' setting in which the number of training examples available is different and not fixed between categories. Experiments are conducted on curated versions of existing datasets (curated such that they exhibit strong long-tail distributions): SUN-397 [13], Places [7], and ImageNet [5]. The performance of the proposed method is demonstrated to be considerably higher than several more adhoc baselines from the literature.


Scene Graph Generation Strategy with Co-occurrence Knowledge and Learnable Term Frequency

Kim, Hyeongjin, Kim, Sangwon, Ahn, Dasom, Lee, Jong Taek, Ko, Byoung Chul

arXiv.org Artificial Intelligence

Scene graph generation (SGG) is an important task in image understanding because it represents the relationships between objects in an image as a graph structure, making it possible to understand the semantic relationships between objects intuitively. Previous SGG studies used a message-passing neural networks (MPNN) to update features, which can effectively reflect information about surrounding objects. However, these studies have failed to reflect the co-occurrence of objects during SGG generation. In addition, they only addressed the long-tail problem of the training dataset from the perspectives of sampling and learning methods. To address these two problems, we propose CooK, which reflects the Co-occurrence Knowledge between objects, and the learnable term frequency-inverse document frequency (TF-l-IDF) to solve the long-tail problem. We applied the proposed model to the SGG benchmark dataset, and the results showed a performance improvement of up to 3.8% compared with existing state-of-the-art models in SGGen subtask. The proposed method exhibits generalization ability from the results obtained, showing uniform performance improvement for all MPNN models.


AMEND: A Mixture of Experts Framework for Long-tailed Trajectory Prediction

Mercurius, Ray Coden, Ahmadi, Ehsan, Shabestary, Soheil Mohamad Alizadeh, Rasouli, Amir

arXiv.org Artificial Intelligence

Accurate prediction of pedestrians' future motions is critical for intelligent driving systems. Developing models for this task requires rich datasets containing diverse sets of samples. However, the existing naturalistic trajectory prediction datasets are generally imbalanced in favor of simpler samples and lack challenging scenarios. Such a long-tail effect causes prediction models to underperform on the tail portion of the data distribution containing safety-critical scenarios. Previous methods tackle the long-tail problem using methods such as contrastive learning and class-conditioned hypernetworks. These approaches, however, are not modular and cannot be applied to many machine learning architectures. In this work, we propose a modular model-agnostic framework for trajectory prediction that leverages a specialized mixture of experts. In our approach, each expert is trained with a specialized skill with respect to a particular part of the data. To produce predictions, we utilise a router network that selects the best expert by generating relative confidence scores. We conduct experimentation on common pedestrian trajectory prediction datasets and show that besides achieving state-of-the-art performance, our method significantly performs better on long-tail scenarios. We further conduct ablation studies to highlight the contribution of different proposed components.


HiLo: Exploiting High Low Frequency Relations for Unbiased Panoptic Scene Graph Generation

Zhou, Zijian, Shi, Miaojing, Caesar, Holger

arXiv.org Artificial Intelligence

Panoptic Scene Graph generation (PSG) is a recently proposed task in image scene understanding that aims to segment the image and extract triplets of subjects, objects and their relations to build a scene graph. This task is particularly challenging for two reasons. First, it suffers from a long-tail problem in its relation categories, making naive biased methods more inclined to high-frequency relations. Existing unbiased methods tackle the long-tail problem by data/loss rebalancing to favor low-frequency relations. Second, a subject-object pair can have two or more semantically overlapping relations. While existing methods favor one over the other, our proposed HiLo framework lets different network branches specialize on low and high frequency relations, enforce their consistency and fuse the results. To the best of our knowledge we are the first to propose an explicitly unbiased PSG method. In extensive experiments we show that our HiLo framework achieves state-of-the-art results on the PSG task. We also apply our method to the Scene Graph Generation task that predicts boxes instead of masks and see improvements over all baseline methods. Code is available at https://github.com/franciszzj/HiLo.


Alleviating the Long-Tail Problem in Conversational Recommender Systems

Zhao, Zhipeng, Zhou, Kun, Wang, Xiaolei, Zhao, Wayne Xin, Pan, Fan, Cao, Zhao, Wen, Ji-Rong

arXiv.org Artificial Intelligence

Conversational recommender systems (CRS) aim to provide the recommendation service via natural language conversations. To develop an effective CRS, high-quality CRS datasets are very crucial. However, existing CRS datasets suffer from the long-tail issue, \ie a large proportion of items are rarely (or even never) mentioned in the conversations, which are called long-tail items. As a result, the CRSs trained on these datasets tend to recommend frequent items, and the diversity of the recommended items would be largely reduced, making users easier to get bored. To address this issue, this paper presents \textbf{LOT-CRS}, a novel framework that focuses on simulating and utilizing a balanced CRS dataset (\ie covering all the items evenly) for improving \textbf{LO}ng-\textbf{T}ail recommendation performance of CRSs. In our approach, we design two pre-training tasks to enhance the understanding of simulated conversation for long-tail items, and adopt retrieval-augmented fine-tuning with label smoothness strategy to further improve the recommendation of long-tail items. Extensive experiments on two public CRS datasets have demonstrated the effectiveness and extensibility of our approach, especially on long-tail recommendation.


Improving Reinforcement Learning for Neural Relation Extraction with Hierarchical Memory Extractor

Wang, Jianing, Su, Chong

arXiv.org Artificial Intelligence

Distant supervision relation extraction (DSRE) is an efficient method to extract semantic relations on a large-scale heuristic labeling corpus. However, it usually brings in a massive noisy data. In order to alleviate this problem, many recent approaches adopt reinforcement learning (RL), which aims to select correct data autonomously before relation classification. Although these RL methods outperform conventional multi-instance learning-based methods, there are still two neglected problems: 1) the existing RL methods ignore the feedback of noisy data, 2) the reduction of training corpus exacerbates long-tail problem. In this paper, we propose a novel framework to solve the two problems mentioned above. Firstly, we design a novel reward function to obtain feedback from both correct and noisy data. In addition, we use implicit relations information to improve RL. Secondly, we propose the hierarchical memory extractor (HME), which utilizes the gating mechanism to share the semantics from correlative instances between data-rich and data-poor classes. Moreover, we define a hierarchical weighted ranking loss function to implement top-down search processing. Extensive experiments conducted on the widely used NYT dataset show significant improvement over state-of-the-art baseline methods.


The origin of intelligent behavior

#artificialintelligence

When I hear news about "AI" these days, what is often meant are methods for pattern recognition and approximations of complex functions, most importantly in the form of Machine Learning. It is true that we have seen impressive applications of Machine Learning systems in a number of different industries such as product personalization, fraud detection, credit risk modeling, insurance pricing, medical image analysis, or self-driving cars. What is the origin of intelligent behavior? Intelligent behavior is the capability of using one's knowledge about the world to make decisions in novel situations: people act intelligently if the use what they know to get what they want. The premise of AI research is that this type of intelligence is fundamentally computational in nature, and that we can therefore find ways to replicate it in machines.